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DETAILED ACTION 

1. This action is responsive to the amendment filed April 3, 2007. 

2. Claims 1-23 remain pending. 

Response to Amendments 

3. Per Applicants' request, Claims 1,18 and 23 have been amended. 

4. The objection to the specification is withdrawn in view of applicants' 
amendments. 

5. The rejection of Claim 23 under 35 U.S.C. § 112, second paragraph is 
withdrawn in view of Applicants' amendments to the claim. 

6. The objection to the drawings is however maintained because the drawings 
filed February 26, 2002 are informal. Formal drawings are required. 

7. The rejection of Claim 23 under 35 U.S.C. § 101 as being directed to functional 
descriptive material NOT claimed as residing on a computer-readable storage medium 
is withdrawn in view of Applicants' amendment to the claim. 

Response to Arguments 

8. The rejection of Claims 18-21 under 35 U.S.C. § 101 as being directed to non- 
statutory subject matter is maintained because Applicants' arguments pursuant to the 
propriety of the rejection of these claims are not persuasive. 

As submitted by AppKcants, the final result of independent claim 18 is the 
identification of a segment of a multimedia data corresponding to a time period of ; 
uniformity during which a numerical value of an attribute of the element of the 
respective streams meets an attribute uniformity threshold and such final 
identification is usefiil, concrete and tangible to identifying the segments of 
multimedia data of interest as recited in the preamble. 
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In response to Applicants' arguments, the examiner respectfully notes the 
following: 

a) if the claimed method does not use the identification step to achieve a real- 
world, useful, concrete and tangible result, the method does not have a practical 
utility; one would wonder as to the practicality of identifying segments of multimedia 
data of interest for the sake of identifying alone; 

b) furthermore claiming only the step of identifying segments of multimedia 
data of interest without any other additional step is considered to be incomplete and 
indefinite, since the claimed step could be combined with any other conceivable steps 
to achieve different results which are useful, concrete and tangible. Such a step of 
identifying segments of multimedia data of interest would not only read on every 
conceivable method of processing multimedia data in the related art but also preempt 
the use of this step, if patented, in any other patentable method. 

9. Applicants' arguments filed April 3, 2007 have been fiiUy considered but they 
are not persuasive. Following is an examiner's response to applicants' arguments, 
a. 102(b) rejection of Claims 1-16 and 18-20 over Zhang: 
Applicants' arguments: 

The Applicant respectfully traverses this § 102(b) rejection of 
independent claims 1,18 and 23, because, among other things, Zhang 
fails to show and teaches away from ^' identifying a time period of 
uniformity > if any, during which the numerical value of the attribute of 
the element of the respective stream meets an attribute uniformity 
threshold" and " identifying a segment of the multimedia data of interest 
corresponding to an identified time period of uniformity " in as complete 
detail as recited in independent claims 1,18 and 23. . . 
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. . . [F]or the temporal segmentation of shots, Zhang is not 
concemed with identifying a time period of uniformity of a segmented 
shot based on a numerical value of an attribute of an element within the 
segmented shot meeting an attribute uniformity threshold as required by 
the aforementioned limitation of independent claims 1,18 and 23, 
Examiner's response : 

Examiner respectfully notes that contrary to Applicants' assertion 
Zhang does identify a time period of uniformity of a segmented shot 
based on a numerical value of an attribute of an element within the 
segmented shot meeting an attribute uniformity threshold. In section 
2.2.3: 

the claimed numerical value of an attribute of an element is met 
by Zhan^s mean and variance o^^ of the differences of a region under 
consideration between frames of a shot; 

the claimed attribute uniformity threshold is met by Zhan£% 

<^^A> c^^Bi 

the claimed time period of uniformity is met by Zhan^s "t." If 
during period t, the mean is below and the variance a^^ below 

a^^, then that period of uniformity (or of minimal changes) in the 
attribute can help determine that the shot is an A shot, i.e., of 
anchorperson A. 

Applicants' arguments : 

Moreover, for the shot classification of segmented shots, Zhang 
actually teaches away from identifying a segment of the news media 
corresponding to an identified time period of uniformity as required by 
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independent claims 1, 18 and 22 [sic]. Specifically, Zhang tt^xhts that 

such an identification process can be ineffective and time consuming. 

See , Zhang at subsection 2.2.2, paragraph 2. 

Examiner^s response : 

In response to Applicants' Z/?^;/^-teaching-away argument, the 

examiner respectfully reproduces below part of the content of the 

subsection and paragraph cited by Applicants: 

Unfortunately, the amount of variation over a set of similarly produced news 
programs may be too high for a fixed set of a priori model images to be effective. Variations 
in both who the anchorpersons are and what they arc wearing from one program to the next 
will probably make matching against a single model very difficult, if not impossible. 
Furthermore, the matching algorithm, itself, can be very time-consuming, given the number 
of shots to which it may have to be applied. Therefore, instead of using predefined model 
images (emphasis added), we have developed an approach to first locate potential 
anchorperson shots using temporal features. We then acquire model images from these 
candidates for model matching. 

According to the above paragraph, Zhang proposes to improve the 
ineffectiveness and time-consuming process of the matching algorithm 
by locating the potential anchorperson shots using temporal features. . 

Therefore, contrary to Applicants' assertion, Zhang does not teach 
away from identifying a segment of the multimedia data corresponding 
to an identified time period of uniformity as required by Claims 1, 18 
and 23. 

The rejection of Claims 1-16 and 18-20 under 35 U.S.C. § 102(b) 
over Zhang is thus maintained. 

With respect to Claims 2-16 and 19-22, which depend respectively from 
base claims 1 and 1 8, the rejections of these claims are also maintained for the 
same reasons discussed in conjunction with the rejection of the base claims. 
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b. Rejection of Claim 17 under 35 U.S.C. § 103(a) as being 
unpatentable over an article entitled "Automatic Parsing of News Video'' 
to Zhang et ali 

Applicants' argument : 

Claim 17 depends from independent claim 1. Therefore, 
dependent claim 17 includes all of the elements and limitations of 
independent claim 1. It is therefore respectfully submitted by the 
Applicants that dependent claim 17 is allowable Zhang [sic] for at least 
the same reason as set forth herein with respect to independent claim 1 
being allowable Zhang [sic]. 
Examiner's response : 

As set forth herein below in section 16, claim 17 is unpatentable 
over Zhang for the reasons discussed therein. Since Claim 17 depends 
from Claim 1, the rejection of Claim 1 is thus incorporated in that of 
Claim 17. The reasons discussed above for maintaining the rejection of 
Claim 1 are also applicable to Claim 17. 

Drawings 

10. The drawings filed February 26, 2002 are objected to by the examiner because 
the drawings are informal. Formal drawings are required in response to this office 
action. 

The objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC § 101 

11. 35 U.S.C. § 101 reads as follows: 
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Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and usefiil improvement thereof, may obtain a patent therefor, subject to the condition and 
requirements of this tide. 

12; Claims 18-21 are rejected under 35 U.S.C. § 101 because the claimed invention 
is directed to non-statutory subject matter. 

The Supreme Court has ruled that to be statutory, a claimed process must either: (A) 
result in a physical transformation for which a practical application is either disclosed in the 
specification or would have been known to a skilled artisan, or (B) be limited to a practical 
application which produces a useful, tangible, and concrete result.. See Diehr, 450 U.S. at 
183-84, 209 USPQ at 6 (quoting Cochran v. Deener , 94 U.S. 780, 787-88 (1876)) ("A 
[statutory] process is a mode of treatment of certain materials to produce a given result. It is 
an act, or a series of acts, performed upon the subject matter to be transformed and reduced 
to a different state or thing. . , . The process requires that certain things should be done with 
certain substances, and in certain order; but the tools to be used in doing this may be of 
secondary consequence."). See also Alappat . 33 F.3d at 1543, 31 USPQ2d at 1556-57 
(quoting Diehr, 450 U.S. at 192, [209 USPQ at 10]). See also id. at 1569, 31 USPQ2d at 
1578-79 (Newman, J., concurring) ("unpatentability of the principle does not defeat 
patentability of its practical applications") (citing O'Reilly , 56 U.S. (15 How.) at 114-19). 
i. "Practical Apphcation of an Abstract Idea" 

While the Supreme Court has ruled that "transformation" is relevant to a section 101 
inquiry, the Coiirt has expressly refused to hold that it is the only test for 
deterrnining patent eligibility. The Federal Circuit has provided further guidance in 
distinguishing between the judicially-created exceptions to patentable subject matter 
and eligible subject matter. The focus of the inquiry is whether the claim, considered 
as a whole, constitutes "a practical application of an abstract idea." State Street . 149 
F.3d at 1373, 47 USPQ2d at 1600. Thus, the question of whether a claim 
encompasses statutory subject matter should not focus on which category of subject 
matter a claim is directed to (e.g., process or machine), "but rather on the essential 
characteristics of the subject matter, in particular its practical utility." State Street , 
149 F.3d at 1375, 47 USPQ2d at 1602; see also AT&T . 172 F.3d at 1360, 50 
USPQ2d at 1453 (focus on section 101 inquiry is "whether the mathematical 
algorithm was applied in a practical manner"). Accordingly, an "abstract idea" when 
practically applied to a useful end is eligible for a patent. State Street . 149 F.3d at 
1374, 47 USPQ2d at 1601 ("a process, machine, manufacture, or composition of 
matter employing a law of nature, natural phenomenon, or abstract idea is patentable 
subject matter even though a law of nature, natural phenomenon, or abstract idea 
would not, by itself, be entitled to such protection.") (emphasis added); see also 
Alappat . 33 F.3d at 1543, 31, USPQ2d at 1556-57 (holding that "certain types of 
mathematical subject matter, standing alone, represent nothing more than abstract 
ideas until reduced to some type of practical application, and thus that subject matter 
is not, in and of itself, entided to patent protection."). 
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Ml "Useful^ Concrete and Tangible Result"' 

In State Street , the Federal Circuit examined some of its prior section 101 
cases, observing that the claimed inventions in those cases were each for a "practical 
application of an abstract idea" because the elements of the invention operated to 
produce a "useful, concrete and tangible result." State Street, 149 F.3d at 1373-74, 47 
USPQ2d at 1601-02, For example, the court in State Street noted that the claimed 
invention in Alappat "constituted a practical application of an abstract idea (a 
mathematical algorithm, formula, or calculation), because it produced 'a useful, 
concrete and tangible thing - the condition of a patient's heart." Id. 

In determining whether the claim is for a "practical application," the focus is 

not on whether the steps taken to achieve a particular result are useful, tangible and 
concrete, but rather that the final result is "useful, tangible and concrete." The 
Federal Circmt further ruled that it is of litde relevance whether a claim is directed to 
a machine or process for the pvupose of a § 101 analysis. AT&T , 172 F.3d at 1358, 
50 USPQ2d at 1451. 

In this instance, it is unclear as to what the final resvilt of the steps taken 
recited in claim 18 (i.e., identifying a time period of uniformity and identifying a 
segment of the multimedia) would be. Will the identifying steps used for 
forming a cumulative, inter-attribute, union of identified and single period, 
determined based on a dotninant attribute with identified and single periods, 
determined based on another respective attribute, the union defining a story 
segment time interval having a start time and an end time (see Claim 22). 
Claim 18 is thus devoid of any final results that are useful, concrete, and 
tangible. 

As a result, Claim 18 and dependent Claims 19-21 are rejected 
under 35 U.S.C. § 101 as being directed to non-statutory subject matter because 
these claims are not for a practical application that produces a useful, concrete 
and tangible result. 



Claim Rejections - 35 USC § 102 
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13. The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 
that form the basis for the rejection under this section made in this Office action: 

A person shaU be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use 
or on sale in this country, more than one year prior to the date of application for patent in the United States. 

14. Claims 1-16, 18-23 are rejected under 35 U.S.C. § 102(b) as being anticipated by 
Zhang et al. ("Zhang"), Automatic Parsing of News Video, May 15, 1994. 

Claims 1, 18 and 23 

Zhang discloses at least an apparatus, a method and a computer program 
embodied on a computer-readable storage medium for identifying segments of multimedia 
data of interest, the multimedia data comprising a stream of at least one of audio, video and text 
elements, the elements having at least one attribute with a numerical value, the attribute being 
indicative of the content of the element (see at least section Introduction, 2nd the computer 
program comprising: 

an intra-attribute uniformity module for identifying a time period of uniformity, if 

any, during which the numerical value of the attribute of the element of the respective stream 

meets an attribute uniformity threshold (see at least section 2.1, 2.2.3 two last ^fs and 

2.24); and 

a module for identifying a segment of the multimedia data of interest corresponding to 
an identified time period of uniformity (see at least 2.2.3, \ before the last ^. 

Claims 2 and 19 

Zhang further discloses wherein the segment identifying module comprises an attribute 
consolidation module for consolidating pairs of identified time periods of uniformity into a single time 
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period of uniformity that temporally comprises the pair of identified time periods of uniformity (see at 
least 2.1, 3^^ 10- - 

Claims 3 and 20 

Zhang further discloses wherein the consolidating of a pair is based on a comparison 
between a time span intervening between the pair and a threshold that is based on the attribute and 
on a characteristic of a predefined thematic collection of data (see at least 2.1, 3^^* T]). 

Claims 4 and 21 

Zhang further discloses wherein the attribute consolidation module identifies a dominant 
attribute based on a comparison between a threshold and a parameter of a time period of uniformity 
identified by the intra-attribute uniformity module (see at least 2.1, 3""^ % the dominant 
attribute being the changed pixel). 

Claims 5 and 22 

Zhang further discloses wherein the segment identifying module further includes an inter- 
attribute merge module for forming a cumulative, inter-attribute , union of identified and single periods, 
if any, determined based on a dominant attribute with identified and single periods, if any, determined 
based on at least one other respective attribute, the union defining a story segment time interval having 
a start time and an end time, at least some cumulations in forming the union being conditional upon 
the existence of an intersection, at least partial, between an identified or single period being 
accumulated and an identified or single period already accumulated in forming the union (see at 
least 2.2.3-5). 
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Claim 6 

Zhang further discloses wherein the inter-attribute merge module indexes the start time 
and end time of the story segment time interval by characteristics of content of a portion of the 
multimedia data that is temporally within the story segment time interval (see at least 2.2.1, 
*'[s]tarting and ending sequences" and 2.2.5). 

Claim 7 

Zhang further discloses comprising a multimedia segment linking module for establishing a link 
among ones of indexed story segment time intervals that meet a segment relatedness criterion (see at 
least section 4). 

Claim 8 

Zhang further discloses wherein said at least one other respective attribute comprises at 
least two attributes, an ordering of attributes by which said cumulative, inter-attribute union is formed 
being determined based on comparisons between thresholds and respective parameters of a time period 
of uniformity identified by the intra-attribute uniformity module (see at least 2.2.3-5). 

Claim 9 

Zhang further discloses wherein the accumulations continue for multiple passes over the 
attributes (see at least 2.2.4, 2""* ^. 

Claim 10 

Zhang further discloses wherein the multimedia data has a genre, and the ordering changes based 
on the genre of the multimedia data on a second pass and subsequent passes, if any (see at least 
2.2.5). 
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Claim 11 

Zhang further discloses wherein said cumulative, inter-attribute union includes identified 
and single periods that temporally intersect an identified or single period determined based on a 
dominant attribute by at least a predetermined ratio of a length of the respective identified or single 
period determined based on the dominant attribute (see at least 2.2.3-5). 

Claim 12 

Zhang further discloses wherein said inter-attribute merge module is configured to form 
an interim union of an identified or single period determined based on a first attribute with an 
identified or single period determined based on a second attribute y the interim union defining a period 
that is accumulated in forming the cumulative, inter-attribute union (see at least 2.2.3-5). 

Claim 13 

Zhang further discloses said at least one other respective attribute comprising at least two 
attributes, an ordering of attributes by which said cumulative, inter-attribute union is formed being 
subject to revision as said stream of elements is processed by said apparatus to identify one of said 
segments of multimedia data of interest (see at least 2.2.5). 

Claim 14 

Zhang further discloses wherein the segment identifying module further includes an inter- 
attribute merge module for forming a story segment time interval that temporally defines a story 
segment comprising content characteristic of a portion of the stream that is located within an identified 
or single period determined based on a dominant attribute (see at least 2.2.3-5 and 2.1, 3'^* ^. 
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Claim 15 

Zhang further discloses wherein the segment identifying module further includes an inter- 
attribute merge module for forming a cumulative^ inter-attribute, union of identified and single periods, 
if any, determined based on a pre-defined, dominant attribute with identified and single periods, if 
any, determined based on at least one other respective attribute, the union defining a story segment 
time interval having a start time and an end time (see at least 2.2.1, "[sjtarting and ending 
sequences," 2.2.3-5 and 2.2.5). 

Claim 16 

Since Claim 16 recites the features of Claims 4 and 5, the same rejections are 
thus applied. 

CMm Rejections - 35 USC § 103 

15. The following is a quotation of the 35 U.S.C. § 103(a) which form the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this tide, if the differences between the subject matter sought to be patented and the prior art are such 
that the subject matter as a whole would have been obvious at the rime the invention was made to a person having 
ordinary skill in the art to which said subject matter pertains. Patentability shall not negatived by the manner in which 
the invention was made. 

16. Claim 17 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Zhang 
et al. ("Zhang"), Automatic Parsing of News Video, May 15, 1994. 

Claim 17 

Zhang does not specifically disclose wherein the attribute comprises a close-caption 
attribute, the stream includes a text element having representative frames that have the close-caption 
attribute, the numerical value comprising a count of a number of close-caption marker elements 
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encountered in one or more consecutive representative frames in said identifying of a time period of 
uniformity. However, Official notice is taken that it is weU-known in the art to add 
closed-captioned text to video shots for the purpose of allowing hearing-challenged 
customers to follow newscast. 

It would have been obvious to a person of ordinary skill in the art at the time 
the invention was made to modify the automatic parsing process of news video taught 
by Zhang to detect closed-captioned text for the purpose discussed above. 

Conclusion 

17. THIS ACTION IS MADE FINAL. Applicant is reminded of die extension 
of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory action 
is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from 
the mailing date of the advisory action. In no event, however, will the statutory 
period for reply expire later than SIX MONTHS from the mailing date of this fmal 
action. 

18. Any inquiry concerning this communication or earUer communications from 
the examiner should be directed to Hoang-Vu "Antony" Nguyen-Ba whose telephone 
number is (571) 272-3701. The examiner can normally be reached on Tuesday-Friday 
from 7:15 am to 5:35 pm. 
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If attempts to reach the examiner are unsuccessful, the examiner's supervisor. 



John Miller can be reached at (571) 272-7353. 

The fax phone number for the organization where this application or 
proceeding is assigned is (571) 273-8300. 

Any inquiry of a general nature or relating to the status of this application 
should be directed to the TC 2600 Group receptionist (571) 272-. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or PubHc PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http:/ /pair-direct.uspto.gov . 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at (866) 217-9197 (toll-free). 
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